Market Intelligence Data Collection from Heterogeneous Sources with Similarity-Based Selection Clustering Technique Using Knowledge Maps: A Heuristic Approach

Business Intelligence (BI) has emerged as one of software solutions that have maximum allocated investments by many organizations for the year 2005. Among various forms and application-based business intelligence, market intelligence (MI) is viewed as a crucial factor for a company to succeed both operationally and strategically in today’s’ competitive environment. Capturing market intelligence data has apparently become easy, especially with the proliferation of the Web. But, this has made data collection more difficult in reality from the system’s point of view, as data sources on the web are voluminous, heterogeneous in terms of structures and semantics, and some part of it may be irrelevant to a specific organizations’ marketing decision-making context, which is the primary premises of market intelligence systems. To address these three specific problems, an algorithm based on similarity measures and multi-dimensional scaling (MDS), which produces hierarchical clusters of knowledge maps from a training data-source set for collecting inputs from heterogeneous sources for capturing market intelligence, is proposed in this paper. The paper illustrates that this algorithm can reduce irrelevant or highly similar data sources for inclusion in the selected data-source repository – represented in the form of clusters of knowledge maps. Therefore, it acts as a similarity-based selection and filtering tool also, with the specific purpose of data collection for MI. Incorporating more advanced techniques for Knowledge maps creation e.g. the Genetic Algorithm-based approaches can further expand this work.